The image processing technology that reconstructs one high-resolution image from multiple low-resolution images having displacements, is called the super-resolution processing (see Non-Patent Document 1), and many technologies have been developed conventionally.
For example, as described in Non-Patent Document 2, the typical super-resolution processing methods such as the ML (Maximum-Likelihood) method, the MAP method (Maximum A Posterior) method and the POCS (Projection Onto Convex Sets) method are proposed.
The ML method is a method that defines an evaluation function as square error between the estimated pixel value from an assumed high-resolution image and the actually observed pixel value, and sets a high-resolution image minimizing the evaluation function as an estimated image. In other words, the ML method is a super-resolution processing method based on the principle of maximum likelihood estimation.
Further, the MAP method is a method that estimates the high-resolution image minimizing an evaluation function which added probability information of the high-resolution image to square error. In other words, the MAP method is a super-resolution processing method that estimates the high-resolution image as an optimization problem maximizing posterior probability by using certain prior information for the high-resolution image.
Moreover, the POCS method is a super-resolution processing method that obtains the high-resolution image by generating simultaneous equations about the pixel values of the low-resolution image and the high-resolution image and then solving the simultaneous equations successively.
All of the above-described super-resolution processing methods have the common features of assuming a high-resolution image (an initial high-resolution image), estimating its pixel value for each pixel of all low-resolution images based on a point spread function (PSF) obtained from a camera model from the assumed high-resolution image and then searching for a high-resolution image by minimizing the difference between the estimated value and the observed pixel value (the observed value). Therefore, these super-resolution processing methods are called reconstruction-based super-resolution processing methods.
All of the above-described reconstruction-based super-resolution processing methods reconstruct a high-resolution image by the super-resolution processing (an iterative reconstruction processing) that needs an initial high-resolution image (an initial image).
By the way, as for the image data (middle images) that is obtained by registering these multiple low-resolution images in a high-resolution image space based on displacements between multiple low-resolution images, its pixel densities become non-uniform due to influences of motions of subjects, the number of used low-resolution images, a displacement detection processing, and a reliable pixel selection processing etc. That is to say, the pixel density of its image data in the high-resolution image space, is different by the pixel position.
In the case of using an image that is generated based on the image data (middle images) having such non-uniform pixel densities as an initial image of a reconstruction-based super-resolution processing method, the super-resolution processing (an iterative reconstruction processing) by an existing reconstruction-based super-resolution processing method, uses constant values of both the weight coefficient of the constraint condition relating to the iterative reconstruction processing and the termination condition of the iterative reconstruction processing that do not depend on the pixel density, without considering that the pixel densities of the middle images which become the base of the initial image are non-uniform.
As a result, by performing the super-resolution processing (the iterative reconstruction processing) for a region of the initial image that corresponds to a region of the image data (the middle image) obtained by registering multiple low-resolution images in the high-resolution image space where the pixel density is high, conversely, that region of the initial image becomes blurred, a problem that the image quality of that region of the initial image becomes degraded than an interpolated image obtained by a simple interpolation processing occurs.
Further, the iterative reconstruction processing will be performed more than required, therefore there is also a problem that the computation cost for reconstructing a high-resolution image becomes large and it is impossible to effectively generate a high-resolution image.
The present invention has been developed in view of the above described circumstances, and an object of the present invention is to provide a parameter control processing apparatus which adaptively controls parameters relating to the image processing depending on pixel densities of the image data (middle images) obtained by registering multiple low-resolution images in the high-resolution image space.
Further, another object of the present invention is to provide an image processing apparatus which is capable of effectively generating a high-resolution image with high image quality by adaptively controlling parameters relating to the iterative reconstruction processing by the parameter control processing apparatus of the present invention and performing the iterative reconstruction processing based on controlled parameters in the case of generating one high-resolution image from multiple low-resolution images by the super-resolution processing (the iterative reconstruction processing) based on a reconstruction-based super-resolution processing method.